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Abstract — In this paper, we examine a well-establislied model 
for HIV wild-type infection. Tlie algoritlim for steepest descent 
mettled for fixed final-time is stated and a modified method 
for free final-time is presented. The first type of cost functional 
considered, seeks to minimize the total time of therapy. An easy 
implementation for this problem suggests that it can be effective 
in the early stages of treatment as well as for individual-based 
studies, due to the "hit first and hit hard" nature of optimal 
control. 

An LQR based cost functional is also presented and the solu- 
tion is found using steepest descent method. It suggests that the 
optimal therapy must remain high until the patient shows signs 
of recovery after which, the therapy gradually decreases. This 
is in line with the biomedical philosophy. Solution to a modified 
problem which includes a weight for total time is approximated 
using the modified algorithm. It shows a considerable drop in 
the total period. We conclude that, a decreased and optimized 
therapy period can help us increase efficiency as well as the 
turnover rate for patient care. 

I. Introduction 

The World health organization (WHO) classifies human 
immunodeficiency virus (HIV) as a pandemic. HIV progresses 
into acquired immunodeficiency syndrom (AIDS) which leads 
to immune system failure. By 2006, AIDS has claimed over 25 
million lives and over 0.6% of the world population is infected 
with HIVH]. 

HIV is responsible for the selective depletion of CZ)4+ 
cells, also known as helper T cells. As they are essential 
to immune regulation, such depletion leads to an adverse 
effect on the immune system functioning 0. Even though 
HIV is rarely fatal by itself, it increases vulnerability towards 
infections and malignancies fJI. There is no known cure for 
HIV/AIDS as the available drug regimens fail to eliminate HIV 
strains in overall population |4|. The antiretroviral treatments 
available are mostly administered in "drug-cocktail" form, 
also known as highly active antiretroviral therapy (HA ART). 
Even though HAART has been highly effective in treating 
HIV/AIDS, it leads to numerous side-effects from hepatitis, 
Uver failure, cardiovascular malfunction, pancreas damage to 
nausea, diarrhea and depression f3l|. In most patients, the ther- 
apy is long-term and therefore, the search for optimal therapy 
stems from the notion of maintaining a balance between the 
disease and drug side-effects to minimize patient suffering. 



Several models have been proposed for different types of 
HIV therapies. Many of these models study the host-pathogen 
reaction for the virus strain HIV-1 through medical studies or 
numerical simulations [3|. McLean and Nowak |5J address 
the appearance of AZT-resistant strains and its effects on 
treatment. Agur fSl and Agur and Cojucaru fT^ examine the 
effects of chemotherapy on uninfected cells. Much of the 
research is also dedicated to acquired infections in an HIV 
patient JS). Compartment infections are studied and modeled 
in and ||9|. Microphage infections are addressed in ifTOl . 
whereas, transient viremia is discussed in ifTTj . 

In this paper, we discuss the wild-type infection model 
given in |3|. This model is a combination of several of the 
previous models, including, lfT2l . |fT3l . and This model 
encapsulates four different therapies : protease inhibitors, HIV 
fusion inhibitors, T cell enhancer and HIV reverse transcrip- 
tase inhibitor. In this paper, we study the treatment with 
protease inhibitors. 

II. HIV Model for Wild-Type Infection 

This paper uses the model given by Stengel |3 1 for wild-type 
infectious HIV. The model is given by four coupled-ODEs and 
their elements are the HIV particles (xi), uninfected Th cells 
(2:2), proviral cells (2:3) and productively infected Th cells 
(0:4) (all in per mm'^). The different therapies are protease 
inhibitors HIV fusion inhibitors (^2), T cell enhancer 

(ms) and HIV reverse transcriptase inhibitor (u^). The model 
is given by the following ODEs, 
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The virus particles (xi) have a death rate of ai. xi infect 
the uninfected T cells (2:2) at rate 02. This is partially blocked 
by HIV fusion inhibitors with efficacy U2. A portion (1-ui) 
of the production of 2:1 from the productively infected cells 



(xi) is blocked by the protease inhibitors with efficacy ui. 
Birth rate of X2 is proportional to 05 and depends on the 
amount of virus present in the system. The conversion of X2 
to proviral Th cells (2:3) occurs at a rate 02 and is targeted 
by HIV reverse transcriptase inhibitor (7/4). X2 has a natural 
death rate of ag and a proliferation rate of 07. Proliferation is 
enhanced by T cell enhancer (M3). Proviral Th cells (x^) are 
produced through infection. .T3 convert into X4 at a rate ag 
and have a death rate of ag. The infected cells (0:4) convert 
into virus (xi) at rate 04 with the ratio 1:03, completing the 
cycle. The parameter values (from IS) and ||T2]) are ai — 2.4, 
02 = 2.4 X 10"^ 03 ^ 1200, 04 = 0.24, 05 = 10, ag = 0.02, 
07 = 0.03, as = 1500, ag = 0.003. 

We define the vectors x = [xi X2 X3 x^]^ and u = 
[ui U2 U3 Ui]^ , then equation (1) can be written as x = 
/(x(i), u{t), t). The initial conditions are given by x(0) = xo. 
We assume that, the drug efficacy for drug i, Ui varies between 
[ui.Ui] and as the efficacies are normalized this is a subset 
of [0, 1]. In this paper, we discuss the therapy with protease 
inhibitors (ui) and take U2 = M3 = U4 = 0. 

III. Optimal Control Theory 

A general optimal control problem can be stated as a 
combination of a system of ordinary differantial equations 
(ODEs), which defines the dynamics of the state variables 
and controls along with a cost functional, which is to be 
minimized. The problem is defined over a time period [to, i/], 
where tf can be predefined or free. Consider a system given 
by X = /(x, u, t), x(0) — Xq and a cost functional given by, 

J ^ IC{Mtf),tf)+ J^^ C{ji{t),u{t),t)dt. 

If the final time is free, another constraint can be put on the 
final state of the system : 'if{x{tf),tf) — 0, where ^I* is a 
smooth function. Hamiltonian for this system is defined by, 
7{(x, u, A, t) = >C(x, u, t) + X{t) f {k, u, t). The necessary con- 
ditions for a minimum are achieved by defining the modified 
cost functional J' using lagrangian multipliers X{t) and p as, 

J = /C(x(i/),i/)+p*(x(i/),t/) + 

^ ' [C{^{t),u{t),t) - X'^ {t){f{^{t),u{t),t) - x)] dt 

The cost functional reaches minimum when the variation, 
6J^ = with respect to all variables, A,p, x, u and x(t/) 
(also, tf, if final-time is free). Using this argument, if x*{t) 
and u*{t) are the optimal state and control vectors for time 
t respectively then we can state the Pontryagin's necessary 
conditions as following, 

1) ^'(x*(t/),t/) = (Final State Constraint) 

2) i* = VaH = f{x.*{t),u*(t),t) (State Equation) 

3) A = — Vx'H (Costate Equation) 

4) Vu-H = (Optimal Control) 

5) x{to) = Xo 

A(*/) = V,(,,)(/C+pTvi,) 

n{tf) = -VtfilC+p'^'ii) (Boundary Conditions) 



As for the sufficient conditions, where one is concerned with 
the local minimality, it is given by V^.H > (Positive 
definite). For global minimality, one must consider all possible 
controls over time [to,tf]. A Hamiltonian convex in u can 
ensure the global minimality. 

If the final time is free, only the first two boundary condi- 
tions are vaUd. Furthermore, if there is no final state constraint, 
the second boundary condition becomes, X{tf) = Vj-f^^j/C. 

One of the most widely used cost functional is the Linear- 
Quadratic Regulator (LQR) given by, 

J = ]^x^{tf)Sx{tf)^j'' \ [xT(t)gx(t) + uT(t)i?u(t)] dt. 

The matrices, S,Q,R are the weights for the final state 
configuration, integral value of state and control, respectively. 
If the final time is free, one may add a scalar T in the integral 
to include the weight for total time. 

A. Algorithm for fixed final-time optimal control problems 

As it can be seen from the necessary conditions, the 
optimal control system comprises a forward state equation 
and a backward costate equation. For nonlinear models the 
algorithms are essentially iterative. One of the very effective of 
these algorithms is the steepest descent method. The solutions 
in this paper are approximated using this algorithm lfT4l . 

1) Approximation starts with an initial guess u(<) = 
u(")(t). 

2) x(°'(i) is integrated numerically over time [t^, tf] using 
control {t) with x(0) = xq as the boundary condi- 
tion. 

3) Similarly, the costate equation A = —V^(o)H is numer- 
ically solved with X{tf) ~ ^xitj)^ as the boundary 
condition. 

4) The gradient Vu(o)'H is evaluated using numerical tech- 
niques and the updated control for the next iteration is 
given by, 

uW(0 = u(o)(0-r(0)V„<o,H. 
Similarly, the k-nl-th iteration in terms of u'^it) is given 

by, 

u('=+l)(i) =uW(<)-T(fc)V„(.,^. 

5) This iterative procedure is repeated until some con- 
vergence criterion such as, ||u'^''+^) — u^^^^H < tol or 
j(k) _ j(k+i) ^ satisfied. 

B. Modified algorithm for free-final time problems 

If the final time is free, the algorithm can be modified to 
encapsulate it by updating the control as well as the grid 
size for numerical integration in every iteration. For a small 
variation Ai/, one can write /S.J — |^At/. For LQR, this 
can be written as. 



u^{tf)Ru{tf)+T] Atf 

i [xT(t/)gx(i/) + u^{tf)Ruitf) + T] Atf 

(2) 

Suppose, one expects to reduce J by 10%, then one can 
change tf using the following equation, 

0.1 



At 



f 



Ak+l) _ Jk) 

5tf 



(k) 

where, ' represents the k-th iteration. Note that, if the 



number of grid points is n and the grid length at the k-th 
iteration is h''''^ then t^*^-* = (n — l)h'^''\ Therefore, at every 
iteration the grid length can be updated as. 
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This condition corresponds to the third boundary condition 
specified in the last section, i.e., |^ = + Vf^, /C. 

However, this algorithm cannot be effective when the problem 
is specified with a final state constraint ($). 

IV. Results and Discussions 
A. Minimum-time problem 

In minimum-time problems, the cost functional is defined 
as J' — tf — to ^ Jf' dt. They are often presented with 
an acceptable set il of final state variables, i.e., the value of 
$ over this set is zero. The problem specified in the next 
paragraph can be solved using the iterative method described 
in the last section by continuing the process until the system 
either enters the set ft or leaves it, but as there is no incentive 
for the control to be any less than its maximum value, another 
way to define these problems is to find the minimum time it 
takes for the system to enter set Using this notion, we can 
solve the problem with an error of the order of the grid size 
used for numerical integration. 

For the system at hand, the set ft is taken as the values 
of virus particles below a certain level, often called the 
detection level. We take the detection level to be 0.05 fTSl. 
The minimum-time problem can be stated as : To find an 
admissible control ul : [0,tf] [uijU]"] and the final time 
tf, which for the system x = /(x(t), u(i), t), x(0) = xq 



minimizes the cost functional J — t 



f 



tn 



tf such that 



the final virion population is below the detection level. Here, 
the boundary conditions are taken as a;i(0) — 30.0, 0:2(0) = 
904,X3(0) = 3.4,X4(0) = 0.46 and uT = 0.9, for 
pharmacokinetic models for HIV suggest that the efficacy 
decays exponentially and a perfect efficacy throughout the 
treatment period is unlikely to be achieved. The solution to 
this problem is presented in Figure [T] and Figure |2] The results 
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Fig. 1. States for min-time problem 
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Fig. 2. Control for min-time problem 



suggest that it takes about 19 weeks for the virus to climb 
down below the detection level. 

This type of method can be used for finding the optimal 
therapy for curable or highly manageable infections or for 
drugs with minimal side-effects, where a full-blast treatment 
is not detrimental, since the main cause of suffering originates 
from the time the infection is active in the system. This method 
can also be useful in an early detection or treatment for the 
disease, as we shall see in the next subsection that the therapy 
remains at its highest under these conditions. The maximum 
efficacy can be averaged over one period of administering 
drugs for a full-blast therapy. The easy implementation of the 
aforementioned procedure leaves room for studies based on 
individual patient scenarios [17|. 

B. Fixed final-time problem 

In this section, as a prerequisite to the free final-time 
problem, we develop a fixed final-time cost functional (after 
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Fig. 3. States for fixed final-time problem 




Fig. 4. Control for fixed final-time problem 
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Fig. 5. States for free final-time problem 




Fig. 6. Control for free final-time problem 



13]) which sheds light on some of the key features of the 
optimal control. We take 2:1(0) = 4.9, 2:2(0) = 904,2:3(0) 
0.34,^4(0) = 0.42 and tf = 500 days. The cost functional is 
given by, 

J = ^^'^{tf)Sx{tf)+J^ ' i [x^(i)gx(t) + u^{t)Ru{t)\ dt, 

where, S, Q and R are diagonal matrices. We wish to design 
a problem that minimized the terminal and integral values of 
proviral T cells (2:3), productively infected cells (2:4) and the 
virus (2:1) as well as the integral value of the control (wi). 
Keeping these goals in mind, we take the weights to be, 

Sn = ^33 = ^44 = 10^ On = O33 = Q44 = 10^ and 
Rii — 0.01 fi\. The solution is approximated by steepest de- 
cent method described in the previous section and is presented 
in Figure [3] and Figure |4] 

The results suggest that the optimal therapy (u*) stays at 



its maximum for about 230 days and then gradually declines 
over the rest of the period. This is in sync with the bio- 
medical philosophy of "hit first and hit hard" |[3l, i.e., to 
treat the disease with high dosage in the early phase of the 
treatment and once a physical recovery takes place, gradually 
decrease the dosage. The physical recovery is represented by 
the uninfected Th cell count, which rises to almost its highest 
in the first 230 days 1 14|. Similarly, the proviral count and the 
productively infected T cells have gone down substantially. 
Simulations run over a larger or smaller periods along with 
varied values of S,Q, and R suggest that the period of high 
efficacy does not vary by much as long as the values of 
R stay relatively low. This suggests that when treating with 
drugs with minimal side effects, the length of this period stays 
approximately the same llT4l . 

This kind of studies of optimal therapy are very common 
and successful, as they reflect the well-accepted conceptions 



of the bio-medical field. They do, however, fail to address 
the time window for which the therapy continues. This is not 
reflected through the drug side-effects, especially, if they are 
considerably high, for the optimal efficacy changes with the 
period of treatment. Therefore, it is important to take time into 
account when trying to minimize the cost functional. 

C. Free final-time problem 

In this section, we present the model for a free final-time 
LQR. We state the optimal control problem as the following. 

To find an optimal control u{ : [0,i/] [0, 1] which, for 
the system x = f{x{t), u{t), t), minimizes the cost functional, 

+ / ' i [x^{t)Qx{t) + u^(i)i?u(t)] dt, 
Jo ^ 

where, the weights S, Q and R serve the same purpose as 
before and the added term Ttf adds a cost for the duration 
of therapy. The values used are, 5ii — 5*33 — 544 = 10^, 
Qii = Q33 = = 103, = 0.01 and T = 0.001. The 
initial state of the system is taken to be the same as for the 
previous problem, i.e., a:i(0) = 4.9, 2:2(0) = 904,2;3(0) ~ 
0.34, X4(0) = 0.42. 

To solve this problem, the algorithm described in section 
2 is implemented. |^ is given by equation |2] and the grid 
length h is updated after every few iterations. As mentioned 
in section 2, this method tries to satisfy one of the boundary 
conditions, namely, 'H{tf) + WtflC = 0. An implementation in 
MatLab-^*^ shows a considerable drop in this value from the 
initial state. The results are presented in Figure |5] and Figure 

m 

As the results suggest, we notice a substantial drop in the 
treatment window from 500 days to about 275 days. However, 
the essential features of the therapy described in the last 
subsection are retained in Figure |6] The optimal therapy stays 
at its highest for the first 120 days and then shows a gradual 
decrease. This, as noted before, coincides with the restored 
T cell count and the decreased adverse element counts in 
the system. As given in the last subsection, this period of 
high efficacy varies (from 230 days to 120 days) when treated 
over a different time window. The state variables show similar 
progression over time under the two optimal therapies. The 
method decreases the total period of therapy while keeping 
the state variables as intact as possible. 

Conclusion 

In this paper, we consider a well established model for 
wild-type HIV infection. A common LQR optimal therapy 
is approximated with the steepest descent method. We, also, 
present a modified algorithm for approximating the solution 
for free final-time problems. 

A minimum-time problem is stated and the solution is 
approximated with a non-iterative method. We conclude that, 
due to the simple implementation and the "hit first and hit 
hard" nature of the optimal therapy, this kind of studies can 



be very useful in the first phase of therapy and also provide an 
advantage for individual-based optimal-therapy considerations. 

The second cost functional is an LQR defined over a 
fixed time period, which indicates similarities of the optimal 
control with biomedical philosophies. The optimal therapy 
shows a period of high efficacy in the first phase and a 
gradual decrease once there is physical progress in patient. A 
modified problem with free final-time is approximated using 
the algorithm mentioned earlier. It shows a considerable drop 
in the total period. An optimized time window for treatment 
can help us treat patients more efficiently. More number of 
patients can be treated if the optimal period for therapy is 
lower. 

The method presented in this paper is quite similar to 
methods already in use. The need for optimizing the therapy 
period is quite essential as a higher or lower time period 
suggests a non-optimal exposure to either the disease or 
therapy. 
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